Eye movements as a model for active inference

Laurent U Perrinet, INT

Acknowledgements:

Rick Adams and Karl Friston @ UCL - Wellcome Trust Centre for Neuroimaging
Mina Aliakbari Khoei, Guillaume Masson and Anna Montagnini - FACETS-ITN Marie Curie Training Network
Jean-Bernard Damasse, Sohir Rahmouni and Laurent Madelain - ANR REM

LAW, Friday, October 14, 2016

(AUTHOR) Hi, I am Laurent Perrinet from (LOGO) the Institute de Neurosciences de la Timone in Marseille, a joint unit from the CNRS and AMU. Using computational models, I am investigating the link between the efficiency of behavioural responses in vision, their underlying neural code and their adaptation to the structure of the world.
(SHOW TITLE - THEME) = In particular, eye movements, that is, motion of the eyeballs targeted at orienting our gaze in the visual field, provide a large repertoire of behaviors at different time scales from fast, reflexive-like saccades to smooth movememnts and even to long-term adaptation.
(SHOW TITLE -OBJECTIVE) today, I will be focusing on the key challenges in modelizing visually guided eye movements and more generally to better understand decision making in biology (the living kingdom). in particular, we will focus on the dynamics of these mechanisms to unravel the sequence of processes which allow the efficient response of motor commands to the environment. As such, my goal here is to show that the solutions proposed by motion perception and EMs provide a generic (and rather unique) framework to finesse models of decision making in the generic theory championned by Karl Friston, active inference (AI)
(ACKNO) this endeavour involves different techniques, tools and... persons. From the head on, I wish to thank the people who collaborated to this project and in particular Rick Adams and Karl Friston and the Wellcome Trust Centre for Neuroimaging for providing the tools for a successful visit / but also Mina, Guillaume and Anna for the great oppportunity to link theories with psychophysical data; Wahiba, Giacomo and Frédéric for the challenging task of linking such models to the neural activity in NHP, and finally Jean-Bernard, Sohir and Laurent Madelain for their essential knowledge in adaptation and reinforcement.

more precisely... what are eye movements and how can they be useful for active inference?

... Eye movements can be of different nature, and I will propose to you now some gentle post-lunch gymnastics by trying to elicit such visually-guided movements with some short movies:

Most simply, saccades are an orienting behavior aiming at catching a specific position in visual space (or vergence movement in depth). if after fixating at the red fixation dot you try to catch the green dot, then you need to do such saccades, which can be very fast (up to 600 deg/s)

When the dot is now moving almong a trajectory, this visual motion can are also responsible for more complex movements such as the reflexive Ocular Following Response or the voluntary Smooth Pursuit EM which aims at stabilizing the visual image on the retina. INterestingly, in this latter case the position of the eye quickly centers on the present, physical position of the dot and we will later see why this is challenging

... Eye movements can be of different nature, and I will propose to you now some gentle post-lunch gymnastics by trying to elicit such visually-guided movements with some short movies:

The Behavioral receptive field

LP and Masson (2012) Bio Behav Reviews, http://invibe.net/LaurentPerrinet/Publications/Masson12

Montagnini, and Masson (2015) in Biologically Inspired Computer Vision

... we have been busy in the last couple of years to make sense of this repertoire of behaviors, in particular in light with the architecture of the brain areas involved in producing these EMs, for instance for pursuit initiation, ...

the Left-side plot (extracted from this review paper) illustrates in primates a sketch of the hierarchy of neuronal population yielding to ocular following responses from the early visual areas (retina, thalamus, primary visual cortex), finessed by processing in motion specific information in downstream sensory areas (MT for speed or MST for global motion) to the brain stem and the oculomotor muscles which tranform this neural activity in a graded motor output. As such, moving stimuli covering the retina are processed by a cascade of neuronal populations, where information is processed through feedforward, lateral and feedback interactions.
in particular, we took advantage of a Bayesian, probabilistic framework to develop an unified theory, the "behavioral receptive field" which encompasses a large range of psychophysical experiments by modeling behavioral responses from population dynamics. Such cascade implements local, context-dependent extraction of motion information…
today, I would like today to focus on a particular problem which will help us unravel the dynamics of decision making: oculomotor delays. Indeed, one challenge for modelling is to understand EMs using AI as a problem of optimal motor control under axonal delays. The central nervous system has to contend with axonal delays, both at the sensory and the motor levels. For instance, in the human visuo-oculomotor system, it takes approximately $ au_s=50~ms$ for the retinal image to reach the visual areas implicated in motion detection, and a further $ au_m=40~ms $ to reach the oculomotor muscles.

* ... As a consequence, for a tennis player ---here (highly trained) Jo-Wilfried Tsonga at Wimbledon--- trying to intercept a passing-shot ball at a (conservative) speed of $20~m.s^{-1}$, the position sensed on the retinal space corresponds to the instant when its image formed on the photoreceptors of the retina and reaches our hypothetical motion perception area behind:

and at this instant, the sensed physical position is lagging behind (as represented here by $ au_s \cdot v 1~m$ ), that is, approximately at $45$ degrees of eccentricity (red dotted line),
while the position at the moment of emitting the motor command will be $.8~m$ ahead of its present physical position ($ au_m \cdot v$).
As a consequence, note that the player's gaze is directed to the ball at its present position (red line), in anticipatory fashion. Optimal control directs action (future motion of the eye) to the expected position (red dashed line) of the ball in the future --- and the racket (black dashed line) to the expected position of the ball when motor commands reach the periphery (muscles). This is obviously an interesting challenge for modelling an optimal control theory.

* As such, during this talk I will first outline one solution to explain how we may perceive a visual motion while compensating for sensory delays

* knowing this solution, we will then use it in AI to propose a complete solution for "eye movements & oculomotor delays"

Khoei, Masson and LP (2016) In press, PLoS CB http://invibe.net/LaurentPerrinet/Publications/KhoeiMassonPerrinet16

a first stimulus moves continuously across the screen along the central horizontal axis. In the FLE, as this moving stimulus reaches the center of the screen, a second stimulus is flashed just above it and in perfect vertical alignment. Despite the fact that the respective horizontal positions of each stimulus are physically identical when the flash occurs, the moving stimulus is most often perceived ahead of the flashed one.

In this paper (show citation), we propose to extend the hypothesis previously proposed by Nihjawan that this effect is caused by the extrapolation of the stimulus' motion to compensate for the neural delay. However, this hypothesis was challenged by other hypothesis that this effect is due to either anatomy (differential latencies) or to the way visual awareness processes the sequence of events (the post-diction from Eagleman)

As a matter of fact, the motion extrapolation hypothesis was challenged because you can notice that the FLE is still present at initiation of the movement but this effect is not seen if the moving dot abruptly stops at the moment of the flash

Here, we propose an extension of the motion extrapolation hypothesis in a probabilistic framework that explicitly uses the precision of sensory and predicted motion. By using numerical simulations, we show that this theory reconciles the different alternative theories and for the different variants of the FLE.

Sensing with delays

Khoei, Masson and LP (2016) In press, PLoS CB http://invibe.net/LaurentPerrinet/Publications/KhoeiMassonPerrinet16

Our model is very simple: Following what Nihjawan called a "diagonal model", we have proposed to extend a motion-based predictive coding model by including the known sensory delay in the prediction-estimation loop
More formally, the estimated state vector $z = {x, y, u, v}$ is composed of the 2D position ($x$ and $y$) and velocity ($u$ and $v$) of a (possibly moving) stimulus. As such, we extend Nijhawan's diagonal model into a classical Markov chain in order to take into account the known neural delay $ au$: At time $t$, information is integrated until time $t- au$, using a Markov chain and a model of state transitions $p(z_t |z_{t-\delta t})$ such that one can infer the state until the last accessible information $p(z_{t- au} | I_{0:(t- au)})$. This information can then be ``pushed'' forward in time by predicting its trajectory from $t- au$ to $t$. Interestingly, we use the same model of state transition at the time scale of the delay, that is, $p(z_t | z_{t- au})$.
Importantly, the model for transition is based on a simple model of the transport of visual information knowing the velocity. Indeed, knowing $z(t-\Delta t)$ we may infer the future position of this motion at time $t$ by using Newton's first law of mechanics. The precision of the state transition (as given by the shaded area) tunes the strength of this prediction as a diffusion parameter.
Such a model can be implemented in a two-layered neural network including a source (input) and a target (predictive) layer~\citep{KaplanKhoei14}. The source layer receives the delayed sensory information and encodes both position and velocity topographically on the different retinotopic maps of each layer. Notice that here, for simplicity, we show only one 2D map of the motions $(x, v)$. Crucially, the delay compensation in this motion-based prediction model, is simply implemented by connecting each source neuron to a predictive neuron corresponding to the corrected position of stimulus $(x+v\cdot au, v)$ in the target layer. The precision of this anisotropic connectivity map can be tuned by the width of convergence from the source to the target populations. We will see in the second part of this talk that this property (that it is implemented in the connectivity) extends to more complex, multi-layered systems which may generalize to more complex trajectories.

Sensing with delays

Khoei, Masson and LP (2016) In press, PLoS CB http://invibe.net/LaurentPerrinet/Publications/KhoeiMassonPerrinet16

Our model is very simple: Following what Nihjawan called a "diagonal model", we have proposed to extend a motion-based predictive coding model by including the known sensory delay in the prediction-estimation loop
More formally, the estimated state vector $z = {x, y, u, v}$ is composed of the 2D position ($x$ and $y$) and velocity ($u$ and $v$) of a (possibly moving) stimulus. As such, we extend Nijhawan's diagonal model into a classical Markov chain in order to take into account the known neural delay $ au$: At time $t$, information is integrated until time $t- au$, using a Markov chain and a model of state transitions $p(z_t |z_{t-\delta t})$ such that one can infer the state until the last accessible information $p(z_{t- au} | I_{0:(t- au)})$. This information can then be ``pushed'' forward in time by predicting its trajectory from $t- au$ to $t$. Interestingly, we use the same model of state transition at the time scale of the delay, that is, $p(z_t | z_{t- au})$.
Importantly, the model for transition is based on a simple model of the transport of visual information knowing the velocity. Indeed, knowing $z(t-\Delta t)$ we may infer the future position of this motion at time $t$ by using Newton's first law of mechanics. The precision of the state transition (as given by the shaded area) tunes the strength of this prediction as a diffusion parameter.
Such a model can be implemented in a two-layered neural network including a source (input) and a target (predictive) layer~\citep{KaplanKhoei14}. The source layer receives the delayed sensory information and encodes both position and velocity topographically on the different retinotopic maps of each layer. Notice that here, for simplicity, we show only one 2D map of the motions $(x, v)$. Crucially, the delay compensation in this motion-based prediction model, is simply implemented by connecting each source neuron to a predictive neuron corresponding to the corrected position of stimulus $(x+v\cdot au, v)$ in the target layer. The precision of this anisotropic connectivity map can be tuned by the width of convergence from the source to the target populations. We will see in the second part of this talk that this property (that it is implemented in the connectivity) extends to more complex, multi-layered systems which may generalize to more complex trajectories.

Results using PBP

Let me show you now the results of our simulations on the standard flash-lag stimulus by first showing the results of a predictive system with no delay compensation. We here simply show the source layer in the prediction model which tries to track moving patterns but do not compensate for the known delay.
We observe that the moving dot is correctly tracked and the flash correctly detected. I have slowed down the speed of the movie to show that relative to each other, the flash and moving dot appear at the same position at the time of the flash.
The situation is different in the motion-based predictive system : in this slowed down movie, at the time the flash appears, the dot's position was predicted ahead of the position of the flash, similarly to what is reported in psychophysics.
Moreover, we have predicted a similar effect with a large range of parameters of the stimulus, such as speed or contrast. But for the moment, let us focus on the reasons why the model shows a similar behavior

Results using MBP

Let me show you now the results of our simulations on the standard flash-lag stimulus by first showing the results of a predictive system with no delay compensation. We here simply show the source layer in the prediction model which tries to track moving patterns but do not compensate for the known delay.
We observe that the moving dot is correctly tracked and the flash correctly detected. I have slowed down the speed of the movie to show that relative to each other, the flash and moving dot appear at the same position at the time of the flash.
The situation is different in the motion-based predictive system : in this slowed down movie, at the time the flash appears, the dot's position was predicted ahead of the position of the flash, similarly to what is reported in psychophysics.
Moreover, we have predicted a similar effect with a large range of parameters of the stimulus, such as speed or contrast. But for the moment, let us focus on the reasons why the model shows a similar behavior

Sensing with delays

Khoei, Masson and LP (2016) In press, PLoS CB http://invibe.net/LaurentPerrinet/Publications/KhoeiMassonPerrinet16

For that, we replot the movies I have just shown by showing for the dot the Histogram of the estimated positions as a function of time for the source layer (Left) and the target layer (right). The left-hand column illustrates the predictive model before delay compensation. The right-hand column illustrates the motion extrapolation model with delay compensation. Histograms of the inferred horizontal positions (blueish bottom panel) and horizontal speed (redish top panel) are shown in columns as a function of time. A darker level corresponds to a higher probability, while a light color corresponds to an unlikely estimation. In particular, we focus on three particular epochs along the trajectory, corresponding to the standard, flash initiated and terminated cycles. The timing of these epochs flashes are indicated by dashed vertical lines. In dark, the physical time and in green the delayed input knowing $ au=100~ms$.
Activity in both models shows three different phases. First, there is a rapid build-up of the precision of the target after the first appearance of the moving dot (at $t=300~ms$). Consistently with the Frölich effect, the beginning of the trajectory is seen ahead of its physical position. During the second phase, the moving dot is correctly tracked as both its velocity and position are correctly inferred. In the source layer, there is no extrapolation and the trajectory follows the delayed trajectory of the dot (green dotted line). In the target layer, motion extrapolation correctly predicts the position at the present time and the position follows the actual physical position of the dot (black dotted line). Finally, the third phase corresponds to motion termination. The moving dot disappears and the corresponding activity vanishes in the source layer at $t=900~ms$. However, between $t=800~ms$ and $t=900~ms$, the dot position was extrapolated and predicted ahead of the terminal position. At $t=900~ms$, while motion information is absent, the position information is still transiently consistent and extrapolated using a broad, centered prior distribution of speeds. Although it is less precise, this position of the dot at flash termination is therefore not perceived as leading the flash.
Interestingly, and thanks to the reviewers of the paper, we could extend our results to the estimation of the dot position from the dMBP model during the motion reversal experiment. In the motion reversal experiment, the moving dot reverses its direction at the middle of the trajectory (i.e., at $t=500~ms$, as indicated by the mid-point vertical dashed line). In the left column (target layer) and as in previous slide, we show the histogram of inferred positions during the dot motion and a trace of its position with the highest probability as a function of time. As expected, results are identical to that in the previous slide in the first half period. At the moment of the motion reversal, the model output is consistent with previous psychophysical reports. First, the estimated position follows the extrapolated trajectory until the (delayed) sensory information about the motion reversal reaches the system (at $t=600~ms$, green vertical dashed line). Then, the velocity is quickly reseted and converges to the new (reversed) motion such that the estimated position ``jumps'' to a position corresponding to the updated velocity. In the right column (smoothed layer), we show the results of the same data after a smoothing operation of $ au_s=100~ms$ in subjective time. This different read-out from the inferred positions corresponds to the behavioral results obtained in some experiments, such as that from~\citet{Whitney98}.

Sensing with delays

Khoei, Masson and LP (2016) In press, PLoS CB http://invibe.net/LaurentPerrinet/Publications/KhoeiMassonPerrinet16

For that, we replot the movies I have just shown by showing for the dot the Histogram of the estimated positions as a function of time for the source layer (Left) and the target layer (right). The left-hand column illustrates the predictive model before delay compensation. The right-hand column illustrates the motion extrapolation model with delay compensation. Histograms of the inferred horizontal positions (blueish bottom panel) and horizontal speed (redish top panel) are shown in columns as a function of time. A darker level corresponds to a higher probability, while a light color corresponds to an unlikely estimation. In particular, we focus on three particular epochs along the trajectory, corresponding to the standard, flash initiated and terminated cycles. The timing of these epochs flashes are indicated by dashed vertical lines. In dark, the physical time and in green the delayed input knowing $ au=100~ms$.
Activity in both models shows three different phases. First, there is a rapid build-up of the precision of the target after the first appearance of the moving dot (at $t=300~ms$). Consistently with the Frölich effect, the beginning of the trajectory is seen ahead of its physical position. During the second phase, the moving dot is correctly tracked as both its velocity and position are correctly inferred. In the source layer, there is no extrapolation and the trajectory follows the delayed trajectory of the dot (green dotted line). In the target layer, motion extrapolation correctly predicts the position at the present time and the position follows the actual physical position of the dot (black dotted line). Finally, the third phase corresponds to motion termination. The moving dot disappears and the corresponding activity vanishes in the source layer at $t=900~ms$. However, between $t=800~ms$ and $t=900~ms$, the dot position was extrapolated and predicted ahead of the terminal position. At $t=900~ms$, while motion information is absent, the position information is still transiently consistent and extrapolated using a broad, centered prior distribution of speeds. Although it is less precise, this position of the dot at flash termination is therefore not perceived as leading the flash.
Interestingly, and thanks to the reviewers of the paper, we could extend our results to the estimation of the dot position from the dMBP model during the motion reversal experiment. In the motion reversal experiment, the moving dot reverses its direction at the middle of the trajectory (i.e., at $t=500~ms$, as indicated by the mid-point vertical dashed line). In the left column (target layer) and as in previous slide, we show the histogram of inferred positions during the dot motion and a trace of its position with the highest probability as a function of time. As expected, results are identical to that in the previous slide in the first half period. At the moment of the motion reversal, the model output is consistent with previous psychophysical reports. First, the estimated position follows the extrapolated trajectory until the (delayed) sensory information about the motion reversal reaches the system (at $t=600~ms$, green vertical dashed line). Then, the velocity is quickly reseted and converges to the new (reversed) motion such that the estimated position ``jumps'' to a position corresponding to the updated velocity. In the right column (smoothed layer), we show the results of the same data after a smoothing operation of $ au_s=100~ms$ in subjective time. This different read-out from the inferred positions corresponds to the behavioral results obtained in some experiments, such as that from~\citet{Whitney98}.

Sensing with delays

Khoei, Masson and LP (2016) In press, PLoS CB http://invibe.net/LaurentPerrinet/Publications/KhoeiMassonPerrinet16

For that, we replot the movies I have just shown by showing for the dot the Histogram of the estimated positions as a function of time for the source layer (Left) and the target layer (right). The left-hand column illustrates the predictive model before delay compensation. The right-hand column illustrates the motion extrapolation model with delay compensation. Histograms of the inferred horizontal positions (blueish bottom panel) and horizontal speed (redish top panel) are shown in columns as a function of time. A darker level corresponds to a higher probability, while a light color corresponds to an unlikely estimation. In particular, we focus on three particular epochs along the trajectory, corresponding to the standard, flash initiated and terminated cycles. The timing of these epochs flashes are indicated by dashed vertical lines. In dark, the physical time and in green the delayed input knowing $ au=100~ms$.
Activity in both models shows three different phases. First, there is a rapid build-up of the precision of the target after the first appearance of the moving dot (at $t=300~ms$). Consistently with the Frölich effect, the beginning of the trajectory is seen ahead of its physical position. During the second phase, the moving dot is correctly tracked as both its velocity and position are correctly inferred. In the source layer, there is no extrapolation and the trajectory follows the delayed trajectory of the dot (green dotted line). In the target layer, motion extrapolation correctly predicts the position at the present time and the position follows the actual physical position of the dot (black dotted line). Finally, the third phase corresponds to motion termination. The moving dot disappears and the corresponding activity vanishes in the source layer at $t=900~ms$. However, between $t=800~ms$ and $t=900~ms$, the dot position was extrapolated and predicted ahead of the terminal position. At $t=900~ms$, while motion information is absent, the position information is still transiently consistent and extrapolated using a broad, centered prior distribution of speeds. Although it is less precise, this position of the dot at flash termination is therefore not perceived as leading the flash.
Interestingly, and thanks to the reviewers of the paper, we could extend our results to the estimation of the dot position from the dMBP model during the motion reversal experiment. In the motion reversal experiment, the moving dot reverses its direction at the middle of the trajectory (i.e., at $t=500~ms$, as indicated by the mid-point vertical dashed line). In the left column (target layer) and as in previous slide, we show the histogram of inferred positions during the dot motion and a trace of its position with the highest probability as a function of time. As expected, results are identical to that in the previous slide in the first half period. At the moment of the motion reversal, the model output is consistent with previous psychophysical reports. First, the estimated position follows the extrapolated trajectory until the (delayed) sensory information about the motion reversal reaches the system (at $t=600~ms$, green vertical dashed line). Then, the velocity is quickly reseted and converges to the new (reversed) motion such that the estimated position ``jumps'' to a position corresponding to the updated velocity. In the right column (smoothed layer), we show the results of the same data after a smoothing operation of $ au_s=100~ms$ in subjective time. This different read-out from the inferred positions corresponds to the behavioral results obtained in some experiments, such as that from~\citet{Whitney98}.

as an interim summary, we have shown that, evolving from other theories of prediction of post-diction, we have shown that perceptual results are well modeled by a simple theory. This theory states that the agent models (predicts) the scene at the present time (in spite of inherent but known delays) - and we called it parodiction from the greek "paros" the present time.

More generally, this novel theory proposes that generic neural computations systematically compensate neural delays to represent the predicted visual scene at the present time, though it is not yet directly experienced.

a consequence of this view -that is shared among a large number of physicists- is that time as a continous line is an emerging property of the representation used by these agents and does not need to exist physically (though this rather bold statement is quite infalsifiable) - the world is latent - there is only the arrow of time (= laws of nature)
There is a fundamental limit to this theory: the agent has an uniform access to the whole field of view. With a foveal point, it may fail to track - thus we need action as an eye movement

in a first study, we have proposed that PERCEPTION (following Helmhotz 1866) is an active process of hypothesis testing by which we seek to confirm our predictive models of the (hidden) world: Active inference: (cite TITLE). In theory, one could find any prior to fit any experimental data, but the beauty of the theory comes from the simplicity of the models chosen to model the data at hand, such as saccades...
... and even better if these models may find a possible correspondance into the neural anatomy and explain some deviation to a control behaviour, such as that we modelled for understanding some aspects of the EMs of schizophrenic patients
Today, I will again focus on the problem of sensorimotor delays in the optimal control of (smooth) eye movements under uncertainty. Specifically, we consider delays in the visuo-oculomotor loop and their implications for active inference and I will present the results presented in the following paper (show TITLE).

This schematic shows the dependencies among various quantities modelling exchanges of an agent with the environment. It shows the states of the environment and the system in terms of a probabilistic dependency graph, where connections denote directed (causal) dependencies. The quantities are described within the nodes of this graph -- with exemplar forms for their dependencies on other variables.
Hidden (external) and internal states of the agent are separated by action and sensory states. Both action and internal states -- encoding a conditional probability density function over hidden states -- minimise free energy. Note that hidden states in the real world and the form of their dynamics can be different from that assumed by the generative model; (this is why hidden states are in bold. )

Pursuit initiation

This figure reports the conditional estimates of hidden states and causes during the simulation of pursuit initiation, using a single rightward (positive) sweep of a visual target, while compensating for sensory motor delays. We will use the format of this figure in subsequent figures: the upper left panel shows the predicted sensory input (coloured lines) and sensory prediction errors (dotted red lines) along with the true values (broken black lines). Here, we see horizontal excursions of oculomotor angle (upper lines) and the angular position of the target in an intrinsic frame of reference (lower lines). This is effectively the distance of the target from the centre of gaze and reports the spatial lag of the target that is being followed (solid red line). One can see clearly the initial displacement of the target that is suppressed after a few hundred milliseconds. The sensory predictions are based upon the conditional expectations of hidden oculomotor (blue line) and target (red line) angular displacements shown on the upper right. The grey regions correspond to 90% Bayesian confidence intervals and the broken lines show the true values of these hidden states. One can see the motion that elicits following responses and the oculomotor excursion that follows with a short delay of about 64ms. The hidden cause of these displacements is shown with its conditional expectation on the lower left. The true cause and action are shown on the lower right. The action (blue line) is responsible for oculomotor displacements and is driven by the proprioceptive prediction errors.

This figure illustrates the effects of sensorimotor delays on pursuit initiation (red lines) in relation to compensated (optimal) active inference -- as shown in the previous figure (blue lines). The left panels show the true (solid lines) and estimated sensory input (dotted lines), while action is shown in the right panels. Under pure sensory delays (top row), one can see clearly the delay in sensory predictions, in relation to the true inputs. The thicker (solid and dotted) red lines correspond respectively to (true and predicted) proprioceptive input, reflecting oculomotor displacement. The middle row shows the equivalent results with pure motor delays and the lower row presents the results with combined sensorimotor delays. Of note here is the failure of optimal control with oscillatory fluctuations in oculomotor trajectories, which become unstable under combined sensorimotor delays.

This figure illustrates the effects of sensorimotor delays on pursuit initiation (red lines) in relation to compensated (optimal) active inference -- as shown in the previous figure (blue lines). The left panels show the true (solid lines) and estimated sensory input (dotted lines), while action is shown in the right panels. Under pure sensory delays (top row), one can see clearly the delay in sensory predictions, in relation to the true inputs. The thicker (solid and dotted) red lines correspond respectively to (true and predicted) proprioceptive input, reflecting oculomotor displacement. The middle row shows the equivalent results with pure motor delays and the lower row presents the results with combined sensorimotor delays. Of note here is the failure of optimal control with oscillatory fluctuations in oculomotor trajectories, which become unstable under combined sensorimotor delays.

This figure illustrates the effects of sensorimotor delays on pursuit initiation (red lines) in relation to compensated (optimal) active inference -- as shown in the previous figure (blue lines). The left panels show the true (solid lines) and estimated sensory input (dotted lines), while action is shown in the right panels. Under pure sensory delays (top row), one can see clearly the delay in sensory predictions, in relation to the true inputs. The thicker (solid and dotted) red lines correspond respectively to (true and predicted) proprioceptive input, reflecting oculomotor displacement. The middle row shows the equivalent results with pure motor delays and the lower row presents the results with combined sensorimotor delays. Of note here is the failure of optimal control with oscillatory fluctuations in oculomotor trajectories, which become unstable under combined sensorimotor delays.

This figure illustrates the effects of sensorimotor delays on pursuit initiation (red lines) in relation to compensated (optimal) active inference -- as shown in the previous figure (blue lines). The left panels show the true (solid lines) and estimated sensory input (dotted lines), while action is shown in the right panels. Under pure sensory delays (top row), one can see clearly the delay in sensory predictions, in relation to the true inputs. The thicker (solid and dotted) red lines correspond respectively to (true and predicted) proprioceptive input, reflecting oculomotor displacement. The middle row shows the equivalent results with pure motor delays and the lower row presents the results with combined sensorimotor delays. Of note here is the failure of optimal control with oscillatory fluctuations in oculomotor trajectories, which become unstable under combined sensorimotor delays.

Smooth Pursuit

We then consider an extension of the generative model to simulate smooth pursuit eye movements --- in which the visuo-oculomotor system believes both the target and its centre of gaze are attracted to a (hidden) point moving in the visual field.

This figure uses the same format as the previous figure -- the only difference is that the target motion has been rectified so that it is (approximately) hemi-sinusoidal. The thing to note here is that the improved accuracy of the pursuit previously apparent at the onset of the second cycle of motion has now disappeared -- because active inference does not have access to the immediately preceding trajectory. This failure of an anticipatory improvement in tracking is contrary to empirical predictions. \item \odot
the generative model has been equipped with a second hierarchical level that contains hidden states, modelling latent periodic behaviour of the (hidden) causes of target motion. With this addition, the improvement in pursuit accuracy apparent at the onset of the second cycle of motion is reinstated. This is because the model has an internal representation of latent causes of target motion that can be called upon even when these causes are not expressed explicitly in the target trajectory.

This figure uses the same format as the previous figure -- the only difference is that the target motion has been rectified so that it is (approximately) hemi-sinusoidal. The thing to note here is that the improved accuracy of the pursuit previously apparent at the onset of the second cycle of motion has now disappeared -- because active inference does not have access to the immediately preceding trajectory. This failure of an anticipatory improvement in tracking is contrary to empirical predictions. \item \odot
the generative model has been equipped with a second hierarchical level that contains hidden states, modelling latent periodic behaviour of the (hidden) causes of target motion. With this addition, the improvement in pursuit accuracy apparent at the onset of the second cycle of motion is reinstated. This is because the model has an internal representation of latent causes of target motion that can be called upon even when these causes are not expressed explicitly in the target trajectory.

Smooth Pursuit with oclusion

Finally, the generative model is equipped with a hierarchical structure, so that it can recognise and remember unseen (occluded) trajectories and emit anticipatory responses.

This figure uses the same format as the previous figure -- the only difference is that the target motion has been rectified so that it is (approximately) hemi-sinusoidal. The thing to note here is that the improved accuracy of the pursuit previously apparent at the onset of the second cycle of motion has now disappeared -- because active inference does not have access to the immediately preceding trajectory. This failure of an anticipatory improvement in tracking is contrary to empirical predictions. \item \odot
the generative model has been equipped with a second hierarchical level that contains hidden states, modelling latent periodic behaviour of the (hidden) causes of target motion. With this addition, the improvement in pursuit accuracy apparent at the onset of the second cycle of motion is reinstated. This is because the model has an internal representation of latent causes of target motion that can be called upon even when these causes are not expressed explicitly in the target trajectory.

This figure uses the same format as the previous figure -- the only difference is that the target motion has been rectified so that it is (approximately) hemi-sinusoidal. The thing to note here is that the improved accuracy of the pursuit previously apparent at the onset of the second cycle of motion has now disappeared -- because active inference does not have access to the immediately preceding trajectory. This failure of an anticipatory improvement in tracking is contrary to empirical predictions. \item \odot
the generative model has been equipped with a second hierarchical level that contains hidden states, modelling latent periodic behaviour of the (hidden) causes of target motion. With this addition, the improvement in pursuit accuracy apparent at the onset of the second cycle of motion is reinstated. This is because the model has an internal representation of latent causes of target motion that can be called upon even when these causes are not expressed explicitly in the target trajectory.

These simulations speak to a straightforward and neurobiologically plausible solution to the generic problem of integrating information from different sources with different temporal delays and the particular difficulties encountered when a system --- like the oculomotor system of JWT --- tries to control its environment with delayed signals.
this illustrates that this theory can be extended to more complex hierarchies : in the near future we will use such technology to "synchronize" the different sensors and effectors but also the internal modules of a drone and optimize its overall control.
more generally an interesting outlook are applications to machine learning : in big data, the challenge is to sort out the "good" data, to focus our gaze at the right chunk
to conclude, we have shown that EMs can be a great model at different time scales and that such models -and notablty the acute use of generalized coordinates- can be essential for theories to test predictive models such as the sensory system (as for the FLE), but also the oculomotor system as shown above
as such, an hypothesis that we will challenge in the context of our biomimetic drone is an interesting extension at a longer time scale how the system can adapt to smooth variability in the sensory motor contingencies [OR is to know why most mammals adopted a fovea and eye movements (catching preys) ]

Eye movements as a model for active inference

Laurent U Perrinet, INT

Acknowledgements:

Rick Adams and Karl Friston @ UCL - Wellcome Trust Centre for Neuroimaging
Mina Aliakbari Khoei, Guillaume Masson and Anna Montagnini - FACETS-ITN Marie Curie Training Network
Jean-Bernard Damasse, Sohir Rahmouni and Laurent Madelain - ANR REM

LAW, Friday, October 14, 2016

Outline

Sensing with delays

Active inference, eye movements & oculomotor delays

Perspectives: Saccadic adaptation

In the saccadic adaptation paradigm, ...

TODO: loop 4 times the standard and stop for a fifth one.

... Let us now look at the resulting eye movements

To study the recalibration process, one has to introduce a change in the sensory-motor mapping. This can be done by using a saccadic adaptation paradigm such as the McLaughlin, or double step, paradigm in which a saccade target is presented and then displaced before the saccade is completed. Prior to adaptation, the displacement introduces an error between the perceived location of the final target position of the double-step stimulus and the muscle command intended to move the eye to the initial target position (Hopp & Fuchs, 2004; McLaughlin, 1967). Within a few trials, the initial saccade amplitude is adjusted to minimize the visual error produced by the displacement. The perceived direction of a flashed saccadic stimulus is also affected by saccade adaptation (Bahcall & Kowler, 1999). We used the double-step paradigm to adapt saccades and a pointing task to investigate how such adaptation affects perceived direction.

In the saccadic adaptation paradigm, ...

TODO: loop 4 times the standard and stop for a fifth one.

... Let us now look at the resulting eye movements

Perspectives: Saccadic adaptation

* unfolding of a saccade

* one block : adaptation

* we are in the process of modeling such behavior with a hierarchical model (Mathys, 2011)

* synthetic saccades: non active vs active

Saccades (as I’ve described before in this very blog) are rapid point to point displacements of gaze. Unless you have a target which is moving that you can follow with your gaze, you make saccades. This is unique to eye movements

Courses of representative saccadic amplitude adaptations for monkeys (A) and humans (B). Note the difference in abscissa scales! (A) Top: Adaptation and recovery of saccades made to 10° horizontal target steps with a 30% backward adaptation step. Saccadic amplitude is plotted as a function of the number of the trial in each direction. Magenta points, data from preadaptation trials. Red points, data from trials in the adapted direction. Green points, data from interleaved trials in the non-adapted direction. Adaptation and recovery data are fit by exponential function blue. Arrow indicates initial rapid recovery phase. Bottom: Representative examples of target (gray dashed lines) and eye (black lines) position for different trials during preadaptation and adaptation. (B) Adaptation and recovery of saccades made to 15° horizontal target steps with a 33% backward adaptation step. (Note that little change occurs during recovery because the target was extinguished deliberately during the saccade.)